A Strategy for Controlling Item Exposure in Multidimensional Computerized Adaptive Testing

While computerized adaptive tests have enjoyed tremendous growth in the past decade, satisfactory solutions for many important problems are still unavailable. Among these problems, a critical one is the control of item exposure rate. Because adaptive algorithms are designed to select optimal items, they tend to choose items with high discriminating power. Thus, these items are selected far more often than other items, which leads to both over-exposure of some parts of the item pool and under-utilization of others. The risk is that over-used items are often compromised. As a result, they create a security problem that could threaten the validity of a test. Chang and Ying (1999) proposed a stratification scheme to control the exposure rate for one-dimensional tests. In this paper we extend their method to multi-dimensional tests. Specifically, we propose a strategy based on stratification in accordance with a functional of the vector of the discrimination parameter. The proposed strategy is conceptually appealing and can be implemented with minimal computational overhead. In the paper, we provide both theoretical and empirical studies to validate the multi-dimensional stratification method. Our empirical results indicate that the proposed method achieves significant improvement over the optimal method of controlling exposure rate and that it requires only a small sacrifice in efficiency. keywords: item response theory, computerized adaptive testing, discrimination parameter, recursive maximum likelihood estimate, test security.